Method and system for enabling information in augmented reality applications

ABSTRACT

A method includes identifies first user-specific rules defining information associated with a first user to make available to one or more other users. The method identifies second user-specific rules defining information of interest associated with a second user. The method detects the first user is in a vicinity of the second user. The method determines based on the first user-specific rules and the second user-specific rules, a set of information associated with the first user to share with the second user. The method displays to the second user in an augmented reality application, the determined set of information associated with the first user.

The present invention relates to the field of digital computer systems,and more specifically, to augmented reality applications.

BACKGROUND

Meeting people in real life can be a struggle since relevant informationabout someone might not be present in the moment of meeting. Relevantinformation such as the person's name, affiliation, and significantcharacteristics are often missing. Augmented reality (AR) and wearablessuch as glasses or augmented reality apps for mobile devices are used toaugment the real view with other information. However, the augmentinginformation needs to be controlled in order to achieve the desiredeffect of the AR applications.

SUMMARY

Embodiments in accordance with the present invention disclose a method,computer program product and computer system for enabling informationsharing in augmented reality applications. First user-specific rulesdefining information associated with a first user to make available toone or more other users are identified. Second user-specific rulesdefining information of interest associated with a second user areidentified. The first user is detected as being in a vicinity of thesecond user. A set of information associated with the first user toshare with the second user is determined based on the firstuser-specific rules and the second user-specific rules. The determinedset of information associated with the first user is displayed to thesecond user in an augmented reality application.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The present invention together with the above-mentioned and otherobjects may best be understood from the following detailed descriptionof the embodiments, but not restricted to the embodiments.

FIG. 1 is a block diagram of an augmented reality system, in accordancewith an embodiment of the present invention.

FIG. 2 is a computerized system, suited for implementing one or moremethod steps, in accordance with an embodiment of the present invention.

FIG. 3 is a flowchart of a method for an AR application, in accordancewith an embodiment of the present invention.

DETAILED DESCRIPTION

The descriptions of the various embodiments of the present invention arepresented for purposes of illustration and are not intended to beexhaustive or limited to the embodiments disclosed. Many modificationsand variations are apparent to those of ordinary skill in the artwithout departing from the scope and spirit of the describedembodiments. The terminology used herein is chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein.

The information utilized for augmenting the content of AR applicationdepends on the context in which it is used. For example, technologicalconferences need a special subset of the data, whereas other situations,such as a party or dating application require different kinds ofinformation. As sources for the person-related data may be manifold(e.g., social networks or corporate directories) and the desiredinformation needs to be retrieved from various sources, it may bechallenging to evaluate and consume the data specifically in the momentwhen a person is met. Therefore, relevant information has to be quicklyextracted, filtered, and prepared for display on the (augmented reality)screen. From the opposite point of view, i.e., from individuals whoseinformation is exposed to others, a method is needed where they canconfigure which information is publicly shared or visible in whichoccasion. The present method enables the selection and ranking of usefuldata tailored to specific events or situations, while ensuring privacyby controlling access to personal data available to others.

The present method allows to fetch information about a (potentiallyunknown) person P (e.g. from a central repository) and present it toanother person (user?) U on his/her augmented reality device. Thepresent method provides means to implement privacy constraints byspecifying rules which encode the information that person P wants toshare publicly about him/her. Furthermore, user U is provided the optionto specify a set of rules that encode which information is currentlyrelevant for U when meeting P in particular contexts, e.g., on aconference.

The present method may enable a controlled and centralized access toinformation for AR applications. The present method may enhance thevalue of displayed information for the user of the AR applicationtailored to specific contexts where the AR application is used.

An AR device comprises a device capable of displaying a real-time viewof a physical, real-world environment while altering elements within thedisplayed view of the environment. The AR device may include and/or becommunicatively coupled to detection means such as one or more cameradevices used to capture a view of the real-world environment and mayfurther include computer software and/or hardware configured to augmentelements of the captured scene. The AR device may comprise an ARapplication that is configured for performing the augmented display. Forexample, the AR application may be configured to control a detectionmeans, such as a video camera, in order to collect and render image dataof a physical scene (whether acquired in real-time or previouslyacquired) in combination with the determined information. The renderedimage data may be image data of a camera or video live streaming. Thevideo live streaming may provide live or real-time video content using acamera of the AR device through a display of the AR device or throughanother display e.g. by broadcasting the real-time video content throughthe Internet. The launching of the AR application may for exampletrigger the video live streaming. For example, the determinedinformation may be overlaid with the real-time video content.

According to one embodiment, the determining of the information of thefirst user to share with the second user comprises: generating anidentifier of the detected first user; querying a server for informationabout the detected first user using the generated identifier, therebycausing the server to identify the information of the first user toshare with the second user based on the first and second rules; andreceiving the information of the first user to share with the seconduser. The identifier is a unique identifier of the first user. The firstand second rules may for example be stored on the server and/or on an ARdevice comprising the AR application. The server may enable acentralized control of AR applications and may thus save processingresources at the AR devices that would otherwise be required todetermine the information to be made available.

According to one embodiment, making the determined information of thefirst user available comprises: overlaying the determined informationwith a video content (e.g. live video content) generated (and beingdisplayed) by the augmented reality application. This embodiment mayenable to display only relevant information while ensuring user privacyas defined by first and second rules.

According to one embodiment, the detecting of the first user isperformed while the video content is being displayed. This may enable areal-time access to information on imaged users. The detection of thefirst user is performed by at least one of a camera and a positionsensor, wherein the identifier is generated by processing information ofat least one of the camera and the position sensor. This may enableobtaining accurate information on the imaged users and may avoid thedisplay of wrong information that may be caused by a non-uniqueidentifier. According to one embodiment, the processing of theinformation received from the camera is performed using a facialrecognition algorithm. This may further increase the accuracy of theobtained information and ensure the ID is also generated in an accuratemanner According to one embodiment, providing user-specific first rulescomprising providing a first user interface and receiving from the firstuser via the first user interface the user-specific first rules.According to one embodiment, providing user-specific second rulescomprising providing a second user interface and receiving from thesecond user via the second user interface the user-specific secondrules. According to one embodiment, the first user interface is thesecond user interface. For example, the first user interface may be partof the server and/or AR device. According to one embodiment, the methodfurther comprises: building or providing a database of information onusers using one or more data sources. The data sources may for examplecomprise social networks, corporate directories, or other sources ofinformation related to users.

The database may be part of the server. In this case, the data gatheringand preparation step can be executed asynchronously which minimizeslatencies when processing queries against it. Also, the resultingarchitecture is more reliable in case a connected data source istemporarily not available.

In another example, the database may be provided as a virtual databasewhich provides access to information about people by dispatching queriesto subqueries of connected data sources (the virtual database refers tocontent in external sources such as Internet, intranet, cloud-basedsources, local stores, etc.). In this case there is no need to replicateat the server information that is available in other sources and thus noupdate anomalies or processing efforts are needed. In addition, nosingle point of interest is provided for possible attackers that try tosteal person-relevant data.

According to one embodiment, the second rules further indicate the orderin which the interesting information is to be provided. For example, thesecond rules may specify how available data should be ranked (e.g., byrelevance) to reduce the amount of available information that needs tobe displayed and reduce display size on small-sized displays often foundin wearables or mobile devices. For example, the information may beoverlaid based on its order e.g., the information with the highestrelevance (e.g. first ranked attribute) may be displayed if the size ofthe display is not large enough to display or overlay all of theinformation. This may further optimize and control the providedinformation in AR applications.

According to one embodiment, the method further comprises receiving amodification of the first and/or second rules, wherein the determiningof information of the first user to share with the second user isperformed using the modified rules. Both the first and second users canmodify the rules and the system asserts that only matching results whichsatisfy both conditions are displayed to the second user (consuminguser). This may further increase the accuracy of the displayed data byusing up-to-date rules.

According to one embodiment, the making available of the determinedinformation comprises overlaying the determined information with databeing displayed by the AR application.

According to one embodiment, the method further comprises receivingfurther second rules at an AR device running the AR application, whereinthe making available of the determined information comprises filteringthe determined information using the further second rules resulting infiltered data and overlaying the filtered data with data being displayedby the AR application.

According to one embodiment, the method further comprises providing anAR device comprising the AR application, wherein the AR device comprisesfurther information on the first user, wherein the making available ofthe determined information comprises combining the determinedinformation and the further information and making available thecombined information in the AR application. This may enrich thedisplayed content by the AR application.

According to one embodiment, the method further comprises before makingavailable the combined information in the AR application, selecting fromthe combined information data that satisfy the second rules, and makingavailable the selected data in the AR application.

FIG. 1 illustrates is a block diagram of an augmented reality system 100(AR system) in accordance with an example of the present disclosure.

The AR system 100 comprises a one or more data sources 101 providinginformation about people or users. The data sources 101 may for examplecomprise social networks, corporate directories, or other sources ofinformation related to users such as users U and P. In one example, theAR devices 105 may be used as further data sources of information ofusers. For example, user U may denote the user role of a user thatconsumes the information about the others while user P may denote therole of users that provide information about themselves.

The AR system 100 further comprises a central server 103. The centralserver 103 may be connected to each of the data sources 101 via one ormore networks 102 for example, the Internet, a local area network, awide area network and/or a wireless network. The central server 103 isconfigured to build a person identification database (PDB) 106. Thebuilding of PDB 106 may be performed by gathering and consolidatinginformation about people from data sources 101. The received informationfrom data sources 101 may for example be further preprocessed e.g., byindexing it. The PDB 106 may or may not be part of the central server103 e.g. the central server 103 may be configured to connect to the PDB106 if it is not part of the central server 103.

The PDB 106 may for example comprise a graph database storinginformation in RDF format. The PDB 106 may be the own database ofcentral server 106. In this case, the data gathering and preparationstep can be executed asynchronously which minimizes latencies whenprocessing queries against it and the resulting architecture is morereliable in case a connected data source 101 is temporarily unavailable.

In another example, the PDB 106 may be provided as a virtual databasewhich enables access to information about people by dispatching queriesto subqueries of connected data sources 101. In this case there is noneed to replicate information that is available in other sources e.g.101 and thus no update anomalies, no processing efforts needed for this.In addition, no single point of interest is provided for possibleattackers that try to steal person-relevant data.

In a further example, a hybrid solution combining the two examples ofPDB 106 may be used. This may for example enable that only hot data setsmight be cached in the central server 106 to increase reliability forthis information and reduce latencies when they are queried. The non-hotdata (e.g., data having a creation date that is older than a predefinedtime period, e.g., 3 years) is kept in the sources 101 and requested ondemand when it is needed.

The central server 103 further comprises a server interface 112. Theserver interface 112 may for example be used for receiving privacy rules107 and interest rules 109, hereinafter referred to collectively as “therules 107” and “the rules 109”. The rules 107 and 109 and the serverinterface 112 may be part of an AR management component 104 of thecentral server 103. In another example, the rules 107 and/or 109 may beprovided via the AR device 105 if the users have access to the AR device105.

The rules 107 may for example be rules defined and provided by the userP. The user P may specify privacy rules R about data that shall beshared with other users, e.g., specifying which information valuesshould be visible by which data source. The rules 107 may indicate whichinformation to make available about users such as user P and U invarious contexts. For example, the information of user P stored in PDBmay be provided in the form of a table having attributes (e.g. age,preferences, address, titles, scientific production, etc.). The rules107 may for example indicate for each of the attributes in which context(e.g. the context may be defined by context information such as an eventID, so that a conference context may have a respective event ID etc.)and/or with which users the values of the respective attributes can beprovided. For example, a rule 107 may indicate that the address of userP would not be provided in the context of a technological conferencewhile it can be provided in the context of a party. The rule 107 may forexample further indicate which users should not receive suchinformation.

The rules 109 may indicate which information is interesting to the userssuch as users P and U in various contexts. The user may specify rules109 about data that shall be displayed about other users, e.g., whichinformation values shall be displayed based on some context informationsuch as event identifiers. For example user U may specify that the he orshe wanted to have information related to the skills of a person, e.g.the rule 109 may indicate that user U wants only to have attributevalues for scientific production and title of other users. The rules 109may further indicate the relevance the data to be provided for user U,e.g. user U may require that the values of attribute age have thehighest relevance followed by address attribute values, etc., so that ifthe display device does not have enough space to display all data, thedata may be displayed following its relevance. For example, display agevalues first and if there is still enough space display address values,etc.

For example, user U may want to see relevant information about another(potentially unknown) user P where user P decided upfront whichinformation shall be shared about him/her based on rules 107 and 109.

The central server 103 may be connected to one or more AR devices 105via one or more networks for example, the Internet, a local areanetwork, a wide area network and/or a wireless network. Each of therules 107 and 109 may be received via the server interface 112 and/orvia AR devices 105.

The AR device 105 comprises an AR application 114. The AR device 105 maybe configured to run the AR application 114. The central server 103 mayhave a connection to each AR device 105 via AR application 114. Forexample, the AR device 105 may enable analytic services by running theAR application 114. The AR application 114 comprises a user interface115, a person identification (PID) software component 117, and an ARservice 116. The user interface 115 may for example be configured toreceive data such as rules 109 from users such as user U. Upon receivingrules 107 and/or 109 by the AR device 105, the AR device 105 may connectto central server 103 e.g. by implementing a predefined datacommunication protocol specified, e.g., in the AR management component104. And the received rules, e.g., 107 and/or 109 may be propagated tothe central server 103.

The AR device 105 further comprises user detection or monitoring means(collectively referred to by 118) such as a GPS sensor 118A and camera118N. The PID component 117 may be configured to receive information(e.g., camera image data, geospatial location data, etc.). Thatinformation may be received by the PID component 117 on a periodic basisor by streaming e.g. while user U is using AR application 114. Uponreceiving the information, the PID component 117 may analyze thereceived information to determine if a user is indicated in the receivedinformation. This may enable the PID component 117 to detect andidentify a specific person such as user P whose data (e.g. image orposition) were collected by the detection means 118. For example, thePID component 117 may determine the unique ID upon detecting thepresence of the user P in the vicinity of the user U. The detection ofthe vicinity of the two users can be performed by an image processingapplication or by the PID component 117 that, for example, detects themovement of objects or users in the collected images.

The PID component 117 may for example be configured to generate andprovide a unique person identifier ID using received information aboutuser P from the detection means 118 while user U is using AR application114 and meets user P (users U and P are in the vicinity of each other).The PID component 117 may further use publically shared information 108(e.g. as provided by the central server 103 or data sources 101) todetermine the person identifier.

The unique person identifier may for example be determined using facerecognition algorithms, spatial filters on GPS locations returningpeople nearby, etc. and the unique ID may be provided with a respectivespecific probability determined based on the technique used to generatethe ID. The AR service 116 may be configured to display informationabout detected users e.g. user P by for example overlaying informationon the detected user P with a camera live stream by camera 118N. Theoverlaid information may be obtained from the central server 103 usingthe unique ID.

Although the PID component 117 and detection means 118 are shown as partof AR device 105, the PID component 117 and detection means 118 may bepart of the central server 103 and/or the AR device 105. The personidentification may thus be implemented by the AR device 105 or thecentral server 103 or by combination of both.

For example, if data from data sources 101 needs to be considered whichis not available on the AR device 105, a server implementation may beused to determine the person identifier. If the AR device 105 has allnecessary information to identify users e.g. using contact informationand possibly receiving location information about the users, thedetection mechanism can be implemented on the AR device. A mixture ofboth may be used, e.g., to increase detection accuracy, as a hybridapproach.

Assuming for example that user U is the owner of AR device 105 and thatthe user U has access to the AR application 114. The AR application 114may for example be running e.g. a live video streaming is beingperformed. While using the AR application 114, the user P moves near orat the vicinity of the user U so that the detection means 118 cancollect information about the user P. For example, the AR application114 is being used by user U when meeting user P. The PID component 117is receiving information (e.g. image data and position data) from thedetection means 118 while the AR application 114 (e.g. live videostreaming) is being used by the user U. The PID component 117 maydetect, using received information, the presence of user P when he orshe is in the vicinity of the user U. The PID component 117 may generatethe unique ID for user P using the received information. This may beperformed by for example using face recognition algorithms, spatialfilters on GPS locations returning people nearby, etc. The unique ID maybe provided with a respective specific probability determined based onthe technique used to generate the ID. For example, the probability mayindicate the probability of the imaged object to belong to a face of theuser or not. For example, the probability may indicate that the uniqueID for user P is provided with, e.g., 80% probability. The centralserver 103 may respond to the query if the probability is higher than agiven threshold, e.g., 30%.

The PID component 117 may generate a person information query 113 inorder to query the central server 103 for information about the detecteduser P. The query 113 may indicate the unique ID of the user P. Thequery 113 may further indicate currently active interest rules 109 ifnew rules 109 of user U have been received and not yet propagated tocentral server 103. The query 113 may further indicate contextinformation such as an event ID. For example, the context informationmay be received from user U via user interface 115. The AR application114 may be configured, upon being launched, to prompt the user U toindicate the context information of the context in which the user U is(currently) involved in. In another example, the AR application 114 maybe configured to determine the context information based on the positionof the user U e.g. if the user U is in the school, the contextinformation may be determined accordingly.

Upon receiving the query 113, the central server 103 may be configuredto filter data of user P from the PDB 106 based on rules 107 and theinformation provided in the query 113. This filtering may result inshared information 108. For example, the shared information 108 maycomprise only information allowed by user P to be accessed by user U inthe context indicated by the context information of the query 113.

The central server 103 may further determine which interesting data ofthe shared information 108 that is relevant (e.g. currently relevant)for user U based on the rules 109. This may result in selecting sharedrelevant information 110 from the shared information 108. For example,if the shared information 108 comprises values of the attributesaddress, age, and title, and the rules 109 indicate that only the agevalues are relevant for user U, only attributes values of the attributeage would be selected from the shared information 108.

The shared relevant information 110 may further be processed by thecentral server 103. The processing may for example comprise ranking byrelevance the data for user U as defined in the rules 109, e.g., theuser U may indicate in rules 109 that the attribute values of attributeage has the highest relevance followed by address attribute values, etc.The processing may further comprise preparing a result set for displayon the AR device 105 and/or optimizing the layout or structure of theinformation on that device and sending the result set back to the ARdevice 105 for display to user U via the AR service 116.

Upon receiving query result set from the central server 103, the ARservice 116 may display the received set on top or overlay informationof the result set with the camera live stream.

The AR system 100 may further provide an information displayoptimization algorithm, which may be part of the central server 103and/or the AR device 105 for further optimizing the content of theresult set. The information display optimization algorithm may beconfigured to optimize the display of the result set based ondevice-specific information of the AR device 105 such as screenresolutions. The optimization may further comprise the filtering of theresult set, e.g., based on the device information about how much datacan be displayed at all, effectively reducing the data volume.

The information display optimization algorithm may be used at thecentral server 103 in order to generate the result set. This may enablethe central server 103 to for example use connection information aboutthe devices 105 to optimize the display information centrally. Acentralized computation of the result sets may save energy on the device105 endpoint, effectively extending their battery lifetime.

Using the information display optimization algorithm at the AR device105 may have the advantage that the central server 103 does not need toknow which types of devices 105 are connected. Optionally, an additionalpost processing step may be applied to the data that shall be displayedvia the AR device 105. For example, user U may have collected additionaldata about user P, e.g., personal address books, local email database,etc. on AR device 105 which is not provided by data sources 101. Thisadditional information might be merged with the result set as receivedfrom the central server 103 (or as further processed by the informationdisplay optimization algorithm) to provide additional informationforming an enhanced result set. In this case, rules 109 may optionallybe applied again on the enhanced result set e.g. to rank information byrelevance.

FIG. 2 represents central server 103 as a general computerized system,suited for implementing method steps as involved in the disclosure. Thegeneral computerized system may represent the structure of the AR deviceas well.

It will be appreciated that the methods described herein are at leastpartly non-interactive, and automated by way of computerized systems,such as servers or embedded systems. In exemplary embodiments though,the methods described herein can be implemented in a (partly)interactive system. These methods can further be implemented in software1012, 1022 (including firmware 1022), hardware (processor) 1005, or acombination thereof. In exemplary embodiments, the methods describedherein are implemented in software, as an executable program, and isexecuted by a special or general-purpose digital computer, such as apersonal computer, workstation, minicomputer, or mainframe computer. Themost general system 1000 therefore includes a general-purpose computer103.

In exemplary embodiments, in terms of hardware architecture, as shown inFIG. 2, the computer 103 includes a processor 1005, memory (main memory)1010 coupled to a memory controller 1015, and one or more input and/oroutput (I/O) devices (or peripherals) 20, 1045 that are communicativelycoupled via a local input/output controller 1035. The input/outputcontroller 1035 can be, but is not limited to, one or more buses orother wired or wireless connections, as is known in the art. Theinput/output controller 1035 may have additional elements, which areomitted for simplicity, such as controllers, buffers (caches), drivers,repeaters, and receivers, to enable communications. Further, the localinterface may include address, control, and/or data connections toenable appropriate communications among the aforementioned components.As described herein the I/O devices 20, 1045 may generally include anygeneralized cryptographic card or smart card known in the art.

The processor 1005 is a hardware device for executing software,particularly that stored in memory 1010. The processor 1005 can be anycustom made or commercially available processor, a central processingunit (CPU), an auxiliary processor among several processors associatedwith the computer 103, a semiconductor-based microprocessor (in the formof a microchip or chip set), a macroprocessor, or generally any devicefor executing software instructions.

The memory 1010 can include any one or combination of volatile memoryelements (e.g., random access memory (RAM, such as DRAM, SRAM, SDRAM,etc.)) and nonvolatile memory elements (e.g., ROM, erasable programmableread only memory (EPROM), electronically erasable programmable read onlymemory (EEPROM), programmable read only memory (PROM). Note that thememory 1010 can have a distributed architecture, where variouscomponents are situated remote from one another, but can be accessed bythe processor 1005.

The software in memory 1010 may include one or more separate programs,each of which comprises an ordered listing of executable instructionsfor implementing logical functions, notably functions involved inembodiments of this invention. In the example of FIG. 2, software in thememory 1010 includes instructions 1012 e.g. instructions to managedatabases such as a database management system. The memory 1010 mayfurther comprise a query optimizer. The query optimizer may compriseinstructions e.g. software instructions that when executed may provide aquery execution plan for executing a given query.

The software in memory 1010 shall also typically include a suitableoperating system (OS) 1011. The OS 1011 essentially controls theexecution of other computer programs, such as possibly software 1012 forimplementing methods as described herein.

The methods described herein may be in the form of a source program1012, executable program 1012 (object code), script, or any other entitycomprising a set of instructions 1012 to be performed. When a sourceprogram, then the program needs to be translated via a compiler,assembler, interpreter, or the like, which may or may not be includedwithin the memory 1010, so as to operate properly in connection with theOS 1011. Furthermore, the methods can be written as an object-orientedprogramming language, which has classes of data and methods, or aprocedure programming language, which has routines, subroutines, and/orfunctions.

In exemplary embodiments, a conventional keyboard 1050 and mouse 1055can be coupled to the input/output controller 1035. Other output devicessuch as the I/O devices 1045 may include input devices, for example butnot limited to a printer, a scanner, microphone, and the like. Finally,the I/O devices 20, 1045 may further include devices that communicateboth inputs and outputs, for instance but not limited to, a networkinterface card (NIC) or modulator/demodulator (for accessing otherfiles, devices, systems, or a network), a radio frequency (RF) or othertransceiver, a telephonic interface, a bridge, a router, and the like.The I/O devices 20, 1045 can be any generalized cryptographic card orsmart card known in the art. The system 1000 can further include adisplay controller 1025 coupled to a display 1030. In exemplaryembodiments, the system 1000 can further include a network interface forcoupling to a network 1065. The network 1065 can be an IP-based networkfor communication between the computer 103 and any external server,client and the like via a broadband connection. The network 1065transmits and receives data between the computer 103 and externalsystems 30, which can be involved to perform part or all of the steps ofthe methods discussed herein. In exemplary embodiments, network 1065 canbe a managed IP network administered by a service provider. The network1065 may be implemented in a wireless fashion, e.g., using wirelessprotocols and technologies, such as WiFi, WiMax, etc. The network 1065can also be a packet-switched network such as a local area network, widearea network, metropolitan area network, Internet network, or othersimilar type of network environment. The network 1065 may be a fixedwireless network, a wireless local area network (LAN), a wireless widearea network (WAN) a personal area network (PAN), a virtual privatenetwork (VPN), intranet or other suitable network system and includesequipment for receiving and transmitting signals.

If the computer 103 is a PC, workstation, intelligent device or thelike, the software in the memory 1010 may further include a basic inputoutput system (BIOS) 1022. The BIOS is a set of essential softwareroutines that initialize and test hardware at startup, start the OS1011, and support the transfer of data among the hardware devices. TheBIOS is stored in ROM so that the BIOS can be executed when the computer103 is activated.

When the computer 103 is in operation, the processor 1005 is configuredto execute software 1012 stored within the memory 1010, to communicatedata to and from the memory 1010, and to generally control operations ofthe computer 103 pursuant to the software. The methods described hereinand the OS 1011, in whole or in part, but typically the latter, are readby the processor 1005, possibly buffered within the processor 1005, andthen executed.

When the systems and methods described herein are implemented insoftware 1012, as is shown in FIG. 2, the methods can be stored on anycomputer readable medium, such as storage 1020, for use by or inconnection with any computer related system or method. The storage 1020may comprise a disk storage such as HDD storage.

The memory 1010 may further comprise the AR management component 104.The external systems 30 may for example comprise the PDB 106.

FIG. 3 is a flowchart for an AR application e.g. 114. The AR application114 may for example being used by user U of FIG. 1 and may trigger acamera or video live stream (real-time streaming).

In step 301, the user U and another user P may be detected at thevicinity of each other. This may, for example, be performed by the PIDcomponent 117 or another image analysis component of the AR deviceanalyzing image data by camera 118N and position data by GPS sensor 118Afor detecting that the user P is near or in the vicinity of the user U.The GPS sensor may, for example, indicate the position of user U, ownerof the AR device 105, and may be used in combination with the positionof the object or user P detected in the image data of the camera 118N todetermine the distance between user U and user P (detected in the imagedata). In another example, the AR device 105 may send positioninformation to the central server 103. The position information may, forexample, comprise at least one of the GPS position of the user U and thedetermined distance to the detected user or object. The central server103 may use the received position information in order to identify auser whose (current) GPS position corresponds to the received positioninformation. And the central server 103 may send at least part of sharedinformation 108 of the identified user that indicates his GPS position.The AR device may then use the received GPS position to determine thedistance in a more accurate manner.

For example, the two users U and P are in the vicinity of each other ifthe distance between the users U and P is smaller than a predefinedmaximum distance, e.g., 2 m. For example, the user U may be the owner ofthe AR device running the AR application e.g. making a video stream of agiven scene and the user P appears in the video at a certain point oftime. As the image data representing an object (e.g., the user P) in theacquired video is identified or detected, the position of the objectrepresented in the image data may be determined.

Upon detecting the users U and P as being in the vicinity of each other,a unique identifier may be determined for the user P using at least oneof the image data of the camera 118N and GPS data, such that the uniqueID can be used by the server 103 to provide information on user P.

In step 303, based on the rules 107 and 109, information 110 of the userP to share with the user U may be determined. For example, informationof the user P may be gathered from one or more sources 101. The gatheredinformation may be filtered using the first and second rules resultingin a filtered or selected data. The determined information is thefiltered or selected data.

In step 305, the determined information of the user P may be madeavailable to the user U in the AR application 114.

For example, steps 301-305 may be performed by the central server 103.In this case, the PID component 117 (e.g. and the detection means 118)may be part of the central server 103 for performing the detecting step301. If for example, the detection means 118 are part of the AR device105, the PID component 117 of the central server 103 may receive fromthe AR device 105 the information collected by the detection means 118in order to perform the detection of step 301. The determinedinformation of user P is made available to the user U by sending by thecentral server 103 the information of user P to the AR device 105 andcausing the AR device to display such information e.g. overlay theinformation with the live stream.

In another example, steps 301-305 may be performed by the AR device 105.In this case, the information 110 of the user P to share with the user Umay be determined by querying the central server for such informationand causing the central server 103 to determine such information basedon the rules 107 and 109 and sending the determined information to theAR device 105. The AR device 105 may display the determined informationby overlaying it with the camera live stream.

In another example, a computerized method for augmented realityapplications may be provided. The method comprises: storinguser-specific first rules defining which information to make availableabout the user in various contexts, storing user-specific second rulesdefining which information is interesting to the user in variouscontexts, detecting a first user and a second user at the vicinity ofeach other in a specific context (e.g. a conference), having access touser-specific information of the first user that is intended to beshared with other users, determining, based on the user-specific firstrules of the first user and user-specific second rules of the seconduser, which user-specific information of the first user to share withinsecond user, making the determined user-specific information of thefirst user available to the second user in an augmented realityapplication.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

1. A method comprising: identifying, by one or more processors, firstuser-specific rules defining information associated with a first user tomake available to one or more other users; identifying, by one or moreprocessors, second user-specific rules defining information of interestassociated with a second user; detecting, by one or more processors, thefirst user is in a vicinity of the second user; determining, by one ormore processors, based on the first user-specific rules and the seconduser-specific rules, a set of information associated with the first userto share with the second user; and displaying, by one or moreprocessors, to the second user in an augmented reality application, thedetermined set of information associated with the first user.
 2. Themethod of claim 1, further comprising: generating, by one or moreprocessors, an identifier of the detected first user; querying, by oneor more processors, a server for information about the detected firstuser using the generated identifier, wherein causing the server toidentify the information of the first user to share with the second userbased on the first and second rules; and receiving, by one or moreprocessors, the information of the first user to share with the seconduser.
 3. The method of claim 1, wherein displaying to the second user inan augmented reality application, the determined set of informationassociated with the first user comprises: overlaying, by one or moreprocessors, the determined set of information with the first user with avideo stream generated by the augmented reality application.
 4. Themethod of claim 3, wherein detecting the first user is in the vicinityof the second user while the video stream is generated by the augmentedreality application.
 5. The method of claim 2, wherein detecting thefirst user is in the vicinity of the second user utilizing a camera anda position sensor, wherein the identifier is generated by processinginformation of at least one of the camera and the position sensor. 6.The method of claim 5, wherein the processing information of the camerais performed utilizing a face recognition algorithm.
 7. The method ofclaim 1, further comprising: responsive to identifying a first userinterface, receiving, by one or more processors, from the first user viathe first user interface the first user-specific rules.
 8. The method ofclaim 7, further comprising: responsive to identifying a second userinterface, receiving, by one or more processors, from the second uservia the second user interface the second user-specific rules.
 9. Themethod of claim 8, wherein the first user interface is the second userinterface.
 10. The method of claim 1, further comprising: receiving, byone or more processors, a modification to the first user-specific rules,wherein the modification indicates the set of information associatedwith the first user to share with the second user.
 11. The method ofclaim 1, further comprising: receiving, by one or more processors, thesecond user-specific rules at an augmented reality device running theaugmented reality application; and determining, by one or moreprocessors, to display the determined set of information associated withthe first user in as an overlay in the augmented reality application onthe augmented reality device.
 12. A computer program product comprising:one or more computer readable storage media and program instructionsstored on at least one of the one or more storage media, the programinstructions comprising: program instructions to identify firstuser-specific rules defining information associated with a first user tomake available to one or more other users; program instructions toidentify second user-specific rules defining information of interestassociated with a second user; program instructions to detect the firstuser is in a vicinity of the second user; program instructions todetermine based on the first user-specific rules and the seconduser-specific rules, a set of information associated with the first userto share with the second user; and program instructions to display tothe second user in an augmented reality application, the determined setof information associated with the first user.
 13. The computer programproduct of claim 12, further comprising program instructions, stored onthe one or more computer readable storage media, which when executed bya processor, cause the processor to: generate an identifier of thedetected first user; query a server for information about the detectedfirst user using the generated identifier, wherein causing the server toidentify the information of the first user to share with the second userbased on the first and second rules; and receive the information of thefirst user to share with the second user.
 14. The computer programproduct of claim 12, wherein program instructions to display to thesecond user in an augmented reality application, the determined set ofinformation associated with the first user comprises, programinstructions, stored on the one or more computer readable storage media,which when executed by a processor, cause the processor to: overlay thedetermined set of information with the first user with a video streamgenerated by the augmented reality application.
 15. The computer programproduct of claim 14, wherein program instructions to detect the firstuser is in the vicinity of the second user while the video stream isgenerated by the augmented reality application.
 16. The computer programproduct of claim 13, wherein program instructions to detect the firstuser is in the vicinity of the second user utilizing a camera and aposition sensor, wherein the identifier is generated by processinginformation of at least one of the camera and the position sensor. 17.The computer program product of claim 16, wherein the processinginformation of the camera is performed utilizing a face recognitionalgorithm.
 18. The computer program product of claim 12, furthercomprising program instructions, stored on the one or more computerreadable storage media, which when executed by a processor, cause theprocessor to: responsive to identifying a first user interface, receivefrom the first user via the first user interface the first user-specificrules.
 19. The computer program product of claim 18, further comprisingprogram instructions, stored on the one or more computer readablestorage media, which when executed by a processor, cause the processorto: responsive to identifying a second user interface, receive from thesecond user via the second user interface the second user-specificrules.
 20. A computer system comprising: one or more computerprocessors; one or more computer readable storage media; and programinstructions stored on the computer readable storage media for executionby at least one of the one or more computer processors, the programinstructions comprising: program instructions to identify firstuser-specific rules defining information associated with a first user tomake available to one or more other users; program instructions toidentify second user-specific rules defining information of interestassociated with a second user; program instructions to detect the firstuser is in a vicinity of the second user; program instructions todetermine based on the first user-specific rules and the seconduser-specific rules, a set of information associated with the first userto share with the second user; and program instructions to display tothe second user in an augmented reality application, the determined setof information associated with the first user.